Inverse transformation using pruning for video coding

ABSTRACT

A method for decoding an encoded video bit stream in a video decoder is provided that includes determining a scan pattern type for a transform block to be decoded, decoding a column position X and a row position Y of a last non-zero coefficient in the transform block from the encoded video bit stream, selecting a column-row inverse transform order when the scan pattern type is a first type, selecting a row-column inverse transform order when the scan pattern type is a second type, and performing one dimensional (1D) inverse discrete cosine transformation (IDCT) computations according to the selected transform order to inversely transform the transform block to generate a residual block.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a division of U.S. patent application Ser. No.17/710,932, filed Mar. 31, 2022 (scheduled to grant as U.S. Pat. No.11,625,452), which is a division of U.S. patent application Ser. No.16/988,949, filed Aug. 10, 2020 (now U.S. Pat. No. 11,301,543), which isa continuation of U.S. patent application Ser. No. 15/688,405, filedAug. 28, 2017 (now U.S. Pat. No. 10,783,217), which is a division ofU.S. patent application Ser. No. 13/470,352, filed May 13, 2012 (nowU.S. Pat. No. 9,747,255), which application claims the benefit of U.S.Provisional Patent Application Ser. No. 61/486,008, filed May 13, 2011,all of which are incorporated herein by reference in their entirety.

BACKGROUND OF THE INVENTION Field of the Invention

Embodiments of the present invention generally relate to inversetransformation using pruning in video coding.

Description of the Related Art

Video compression, i.e., video coding, is an essential enabler fordigital video products as it enables the storage and transmission ofdigital video. In general, video compression techniques applyprediction, transformation, quantization, and entropy coding tosequential blocks of pixels in a video sequence to compress, i.e.,encode, the video sequence. Video decompression techniques generallyperform the inverse of these operations in reverse order to decompress,i.e., decode, a compressed video sequence.

Two dimensional (2D) block transforms, e.g., 2D discrete cosinetransforms (DCT), and variants are used in video coding to reducespatial redundancy and achieve compression. Accordingly, 2D inversetransforms, e.g., 2D inverse DCT (IDCT) are performed in video decodingas part of decompressing encoded video. A 2D IDCT is a separabletransform that may be split into row and column one-dimensional (1D)IDCTs for application. The video coding standard in use typicallydefines the order in which the row and column IDCTs are applied so thatan encoded video bit stream is decoded identically in all compliantdecoders. For example, in the H.264/AVC video coding standard, the rowinverse transform is applied first followed by the column inversetransform.

The high frequency region in transform blocks is typically zero due toquantization and the energy compaction properties of the transform. Theknowledge that a large portion of a transform block may be zero isexploited for IDCT pruning, also referred to as partial inversetransformation, to reduce the computational complexity of an IDCT. InIDCT pruning, many 2D IDCT computations that have zero input and zerooutput, i.e., computations corresponding to a region having only zerovalues, may be eliminated to reduce computational complexity. IDCTpruning is a well known technique that is supported by existing videocoding standards that use a zigzag scan pattern to scan coefficients ina transform block.

SUMMARY

Embodiments of the present invention relate to methods and apparatus forinverse transformation using pruning in video coding. In one aspect, amethod for decoding an encoded video bit stream in a video decoder isprovided that includes determining a scan pattern type for a transformblock to be decoded, decoding a column position X and a row position Yof a last non-zero coefficient in the transform block from the encodedvideo bit stream, selecting a column-row inverse transform order whenthe scan pattern type is a first type, selecting a row-column inversetransform order when the scan pattern type is a second type, andperforming one dimensional (1D) inverse discrete cosine transformation(IDCT) computations according to the selected transform order toinversely transform the transform block to generate a residual block.

In one aspect, a method for decoding an encoded video bit stream in avideo decoder is provided that includes decoding a pruned transform flagin the encoded video bit stream, wherein the pruned transform flagindicates that a subset region of a transform block contains allnon-zero coefficients of the transform block, wherein the subset regionconsists of M rows and N columns of the transform block, and inversescanning only quantized transform coefficients of the subset regiondecoded from the encoded video bit stream to generate the transformblock.

In one aspect, a method for pruning an inverse discrete cosinetransformation (IDCT) of a transform block is provided that includesreceiving a transform coefficient and a corresponding IDCT coefficient,setting a prune flag to indicate pruning when a value of the transformcoefficient is zero, setting the prune flag to indicate no pruning whenthe value of the transform coefficient is non-zero, disabling latchingof the transform coefficient for multiplication with the IDCTcoefficient during a clock cycle and outputting a zero when the pruneflag is set to indicate pruning, and enabling latching of the transformcoefficient for multiplication with the IDCT coefficient during a clockcycle and outputting a product of the transform coefficient and the IDCTcoefficient when the pruning flag is set to indicate no pruning

BRIEF DESCRIPTION OF THE DRAWINGS

Particular embodiments will now be described, by way of example only,and with reference to the accompanying drawings:

FIG. 1 is an example of prior art IDCT pruning for a zigzag scanpattern;

FIG. 2 is a block diagram of a digital system;

FIG. 3 is a block diagram of a video encoder;

FIG. 4 is a block diagram of a video decoder;

FIG. 5 is a flow diagram of a method for IDCT pruning;

FIGS. 6-11 are examples of IDCT pruning;

FIGS. 12A and 12B are flow diagrams of methods;

FIG. 13 is a block diagram of an example IDCT pruning architecture;

FIG. 14 is a flow diagram of a method for IDCT pruning; and

FIG. 15 is a block diagram of an illustrative digital system.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

Specific embodiments of the invention will now be described in detailwith reference to the accompanying figures. Like elements in the variousfigures are denoted by like reference numerals for consistency.

As used herein, the term “picture” may refer to a frame or a field of aframe. A frame is a complete image captured during a known timeinterval. The Joint Collaborative Team on Video Coding (JCT-VC) of ITU-TWP3/16 and ISO/IEC JTC 1/SC 29/WG 11 is currently developing thenext-generation video coding standard referred to as High EfficiencyVideo Coding (HEVC). HEVC is expected to provide around 50% improvementin coding efficiency over the current standard, H.264/AVC, as well aslarger resolutions and higher frame rates. For convenience ofdescription, embodiments of the invention are described herein inreference to HEVC. One of ordinary skill in the art will understand thatembodiments of the invention are not limited to HEVC.

Various versions of HEVC are described in the following documents, whichare incorporated by reference herein: T. Wiegand, et al., “WD3: WorkingDraft 3 of High-Efficiency Video Coding,” JCTVC-E603, JointCollaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 andISO/IEC JTC1/SC29/WG11, Geneva, CH, Mar. 16-23, 2011 (“WD3”), B. Bross,et al., “WD4: Working Draft 4 of High-Efficiency Video Coding,JCTVC-F803_d6, Joint Collaborative Team on Video Coding (JCT-VC) ofITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, Torino, IT, Jul. 14-22, 2011(“WD4”), B. Bross. et al., “WD5: Working Draft 5 of High-EfficiencyVideo Coding, JCTVC-G1103_d9, Joint Collaborative Team on Video Coding(JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, Geneva, CH, Nov.21-30, 2011 (“WD5”), and B. Bross, et al., “High Efficiency Video Coding(HEVC) Text Specification Draft 6, JCTVC-H1003, Joint Collaborative Teamon Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG1,Geneva, CH, Nov. 21-30, 2011 (“HEVC Draft 6”).

In HEVC, a largest coding unit (LCU) is the base unit used forblock-based coding. A picture is divided into non-overlapping LCUs. Thatis, an LCU plays a similar role in coding as the macroblock ofH.264/AVC, but it may be larger, e.g., 32×32, 64×64, etc. An LCU may bepartitioned into coding units (CU). A CU is a block of pixels within anLCU and the CUs within an LCU may be of different sizes. Thepartitioning is a recursive quadtree partitioning. The quadtree is splitaccording to various criteria until a leaf is reached, which is referredto as the coding node or coding unit. The maximum hierarchical depth ofthe quadtree is determined by the size of the smallest CU (SCU)permitted. The coding node is the root node of two trees, a predictiontree and a transform tree. A prediction tree specifies the position andsize of prediction units (PU) for a coding unit. A transform treespecifies the position and size of transform units (TU) for a codingunit. A transform unit may not be larger than a coding unit and the sizeof a transform unit may be, for example, 4×4, 8×8, 16×16, 32×32, 4×16,16×4, 8×32, and 32×8. The sizes of the transforms units and predictionunits for a CU are determined by the video encoder during predictionbased on minimization of rate/distortion costs.

Some aspects of this disclosure have been presented to the JCT-VC in M.Budagavi, “IDCT Pruning”, JCTVC-E386, Joint Collaborative Team on VideoCoding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, Geneva,CH, Mar. 16-23, 2011, and M. Budagavi and V. Sze, “IDCT Pruning and ScanDependent Transform Order”, JCTVC-F236, Joint Collaborative Team onVideo Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11,Torino, IT, Jul. 14-22, 2011, which are incorporated by reference hereinin their entirety.

As previously discussed, inverse transformation with pruning is awell-known technique supported by existing video coding standards suchas H.264/AVC. These video coding standards primarily use a zigzag scanpattern to scan coefficients in a transform block and the largesttransform size used is 8×8. Further, the 2D IDCT needed to inverselytransform a block of transform coefficients may be split into row andcolumn one-dimensional (1D) IDCTs for application. The video codingstandard in use typically defines the order in which the row and columnIDCTs are applied in the decoder. For row-column order, a 1D IDCT isperformed on the rows of the transform block, and a 1D DCT is performedis then performed on the columns of the resulting block. For column-roworder, a 1D IDCT is performed on the columns of the transform block, anda 1D DCT is performed is then performed on the rows of the resultingblock.

In general, to perform IDCT pruning when a zigzag scan pattern is used,the decoder determines the position of the last non-zero coefficient ina transform block. This position may be communicated to the decoder inthe encoded bit stream being decoded. Given the position on the lastnon-zero coefficient, the decoder can determine how many rows (orcolumns) of the transform block have at least one non-zero coefficient.For example, assuming a square transform block, for most possiblepositions of the last non-zero coefficient, the sum of the coordinates(x,y) of the position relative to the upper left corner (0,0) of thetransform block added to 1 is the number of rows (or columns) L, i.e.,x+y+1=L, with at least one non-zero coefficient.

Because the remaining rows (columns) contain only zeroes, there is noneed to perform the first 1D IDCT on those rows (columns) as thecomputation results for those rows (columns) will be zero. Accordingly,for row-column order, the computations of the 1D IDCT are performed onlyon the first L rows of the transform block and the remaining rows of theresulting block are assumed to be zero. Similarly, for column-row order,the computations of the 1D IDCT are performed only on the first Lcolumns of the transform block and the remaining columns of theresulting block are assumed to be zero.

FIG. 1 shows an example of IDCT pruning when a zigzag scan pattern isused for a transform block and row-column order for the IDCT is used.The non-shaded regions of each block indicate the region of the blockcontaining only zero coefficients. In this example, the input transformblock 100 is a 16×16 block. The position of the last non-zerocoefficient in the block, indicated by the X, is (4,4). Thus, the numberof rows with at least one non-zero coefficient is x+y+1=4+4=9. The last7 rows are all zero. The first nine rows of the transform block aretransformed using the 1D IDCT to generate the interim results block 102.Only the first 9 rows of the interim results block 102 will havenon-zero values. Then, all of the columns of the interim results block102 are transformed using the 1D IDCT to generate the final inverselytransformed output block 104. In this case of a decoder, the block 104is a block of residual values.

HEVC includes large size 2D transforms, e.g., 16×16 and 32×32, toachieve improved compression performance. The large transforms have highcomputational complexity so techniques such as pruning are needed toreduce this computational complexity. HEVC also supports multiple scanpatterns for scanning of transform blocks for entropy coding. Forexample, in HEVC Draft 6, horizontal, vertical, and fixed up right scanpatterns are specified. A detailed description of when these scanpatterns are to be used may be found in HEVC Draft 6. Zigzag and fixeddown left scan patterns have also been considered, although notcurrently specified in HEVC.

In general, different scan patterns are provided for intra-coded blockswhere there is strong directionality in the residual depending on theintra-prediction direction. For example, when the intra-predictiondirection is horizontal, only the first few rows of the transform blockwill be non-zero, so scanning along rows will provide better compressionefficiency than other modes. In another example, zigzag scanning mayprovide better compression efficiency when there is little or nodirectionality left in the residual block but rather the residual isconcentrated in the low frequency region of the transform, i.e., isconcentrated around the top left corner of the transform block.

Embodiments of the invention provide for techniques for IDCT pruningwhen horizontal, vertical, fixed down left, and fixed up right scanpatterns are used. In some embodiments, a scan adaptive transform order(row-column order or column-row order) is provided that selects thetransform order based on the scan pattern type. In some embodiments, apruned transform flag is provided, that when set by the encoder,indicates that a subset region of a particular size (e.g., 4×4, 8×8,8×16, etc.) contains all of the non-zero coefficients in a transformblock and the remainder of the block is all zeroes. The flag is thenused by a decoder to determine the number of rows (columns) to betransformed for IDCT pruning. In some embodiments, a hardwarearchitecture for IDCT pruning is provided that uses pruning informationto disable computations that have a zero input and a zero output.

FIG. 2 shows a block diagram of a digital system that includes a sourcedigital system 200 that transmits encoded video sequences to adestination digital system 202 via a communication channel 216. Thesource digital system 200 includes a video capture component 204, avideo encoder component 206, and a transmitter component 208. The videocapture component 204 is configured to provide a video sequence to beencoded by the video encoder component 206. The video capture component204 may be, for example, a video camera, a video archive, or a videofeed from a video content provider. In some embodiments, the videocapture component 204 may generate computer graphics as the videosequence, or a combination of live video, archived video, and/orcomputer-generated video.

The video encoder component 206 receives a video sequence from the videocapture component 204 and encodes it for transmission by the transmittercomponent 208. The video encoder component 206 receives the videosequence from the video capture component 204 as a sequence of pictures,divides the pictures into largest coding units (LCUs), and encodes thevideo data in the LCUs. The video encoder component 206 may beconfigured to apply IDCT pruning techniques during the encoding processas described herein. An embodiment of the video encoder component 206 isdescribed in more detail herein in reference to FIG. 3 .

The transmitter component 208 transmits the encoded video data to thedestination digital system 202 via the communication channel 216. Thecommunication channel 216 may be any communication medium, orcombination of communication media suitable for transmission of theencoded video sequence, such as, for example, wired or wirelesscommunication media, a local area network, or a wide area network.

The destination digital system 202 includes a receiver component 210, avideo decoder component 212 and a display component 214. The receivercomponent 210 receives the encoded video data from the source digitalsystem 200 via the communication channel 216 and provides the encodedvideo data to the video decoder component 212 for decoding. The videodecoder component 212 reverses the encoding process performed by thevideo encoder component 206 to reconstruct the LCUs of the videosequence. The video decoder component 212 may be configured to applyIDCT pruning techniques during the decoding process as described herein.An embodiment of the video decoder component 212 is described in moredetail below in reference to FIG. 4 .

The reconstructed video sequence is displayed on the display component214. The display component 214 may be any suitable display device suchas, for example, a plasma display, a liquid crystal display (LCD), alight emitting diode (LED) display, etc.

In some embodiments, the source digital system 200 may also include areceiver component and a video decoder component and/or the destinationdigital system 202 may include a transmitter component and a videoencoder component for transmission of video sequences both directionsfor video steaming, video broadcasting, and video telephony. Further,the video encoder component 206 and the video decoder component 212 mayperform encoding and decoding in accordance with one or more videocompression standards. The video encoder component 206 and the videodecoder component 212 may be implemented in any suitable combination ofsoftware, firmware, and hardware, such as, for example, one or moredigital signal processors (DSPs), microprocessors, discrete logic,application specific integrated circuits (ASICs), field-programmablegate arrays (FPGAs), etc.

FIG. 3 shows a block diagram of the LCU processing portion of an examplevideo encoder. A coding control component (not shown) sequences thevarious operations of the LCU processing, i.e., the coding controlcomponent runs the main control loop for video encoding. The codingcontrol component receives a digital video sequence and performs anyprocessing on the input video sequence that is to be done at the picturelevel, such as determining the coding type (I, P, or B) of a picturebased on the high level coding structure, e.g., IPPP, IBBP,hierarchical-B, and dividing a picture into LCUs for further processing.The coding control component also may determine the initial LCU CUstructure for each CU and provides information regarding this initialLCU CU structure to the various components of the video encoder asneeded. The coding control component also may determine the initial PUand TU structure for each CU and provides information regarding thisinitial structure to the various components of the video encoder asneeded.

The LCU processing receives LCUs of the input video sequence from thecoding control component and encodes the LCUs under the control of thecoding control component to generate the compressed video stream. TheCUs in the CU structure of an LCU may be processed by the LCU processingin a depth-first Z-scan order. The LCUs 300 from the coding control unitare provided as one input of a motion estimation component 320, as oneinput of an intra-prediction component 324, and to a positive input of acombiner 302 (e.g., adder or subtractor or the like). Further, althoughnot specifically shown, the prediction mode of each picture as selectedby the coding control component is provided to a mode selector componentand the entropy encoder 334.

The storage component 318 provides reference data to the motionestimation component 320 and to the motion compensation component 322.The reference data may include one or more previously encoded anddecoded CUs, i.e., reconstructed CUs.

The motion estimation component 320 provides motion data information tothe motion compensation component 322 and the entropy encoder 334. Morespecifically, the motion estimation component 320 performs tests on CUsin an LCU based on multiple inter-prediction modes (e.g., skip mode,merge mode, and normal or direct inter-prediction) and transform blocksizes using reference picture data from storage 318 to choose the bestmotion vector(s)/prediction mode based on a rate distortion coding cost.To perform the tests, the motion estimation component 320 may begin withthe CU structure provided by the coding control component. The motionestimation component 320 may divide each CU indicated in the CUstructure into PUs according to the unit sizes of prediction modes andinto transform units according to the transform block sizes andcalculate the coding costs for each prediction mode and transform blocksize for each CU. The motion estimation component 320 may also computeCU structure for the LCU and PU/TU partitioning structure for a CU ofthe LCU by itself.

For coding efficiency, the motion estimation component 320 may alsodecide to alter the CU structure by further partitioning one or more ofthe CUs in the CU structure. That is, when choosing the best motionvectors/prediction modes, in addition to testing with the initial CUstructure, the motion estimation component 320 may also choose to dividethe larger CUs in the initial CU structure into smaller CUs (within thelimits of the recursive quadtree structure), and calculate coding costsat lower levels in the coding hierarchy. If the motion estimationcomponent 320 changes the initial CU structure, the modified CUstructure is communicated to other components that need the information.

The motion estimation component 320 provides the selected motion vector(MV) or vectors and the selected prediction mode for eachinter-predicted PU of a CU to the motion compensation component 322 andthe selected motion vector (MV), reference picture index (indices),prediction direction (if any) to the entropy encoder 334

The motion compensation component 322 provides motion compensatedinter-prediction information to the mode decision component 326 thatincludes motion compensated inter-predicted PUs, the selectedinter-prediction modes for the inter-predicted PUs, and correspondingtransform block sizes. The coding costs of the inter-predicted PUs arealso provided to the mode decision component 326.

The intra-prediction component 324 provides intra-prediction informationto the mode decision component 326 that includes intra-predicted PUs andthe corresponding intra-prediction modes. That is, the intra-predictioncomponent 324 performs intra-prediction in which tests based on multipleintra-prediction modes and transform unit sizes are performed on CUs inan LCU using previously encoded neighboring PUs from the buffer 328 tochoose the best intra-prediction mode for each PU in the CU based on acoding cost.

To perform the tests, the intra-prediction component 324 may begin withthe CU structure provided by the coding control. The intra-predictioncomponent 324 may divide each CU indicated in the CU structure into PUsaccording to the unit sizes of the intra-prediction modes and intotransform units according to the transform block sizes and calculate thecoding costs for each prediction mode and transform block size for eachPU. For coding efficiency, the intra-prediction component 324 may alsodecide to alter the CU structure by further partitioning one or more ofthe CUs in the CU structure. That is, when choosing the best predictionmodes, in addition to testing with the initial CU structure, theintra-prediction component 324 may also chose to divide the larger CUsin the initial CU structure into smaller CUs (within the limits of therecursive quadtree structure), and calculate coding costs at lowerlevels in the coding hierarchy. If the intra-prediction component 324changes the initial CU structure, the modified CU structure iscommunicated to other components that need the information. Further, thecoding costs of the intra-predicted PUs and the associated transformblock sizes are also provided to the mode decision component 326.

The mode decision component 326 selects between the motion-compensatedinter-predicted PUs from the motion compensation component 322 and theintra-predicted PUs from the intra-prediction component 324 based on thecoding costs of the PUs and the picture prediction mode provided by themode selector component. The decision is made at CU level. Based on thedecision as to whether a CU is to be intra- or inter-coded, theintra-predicted PUs or inter-predicted PUs are selected, accordingly.

The output of the mode decision component 326, i.e., the predicted PU,is provided to a negative input of the combiner 302 and to a delaycomponent 330. The associated transform block size is also provided tothe transform component 304. The output of the delay component 330 isprovided to another combiner (i.e., an adder) 338. The combiner 302subtracts the predicted PU from the current PU to provide a residual PUto the transform component 304. The resulting residual PU is a set ofpixel difference values that quantify differences between pixel valuesof the original PU and the predicted PU. The residual blocks of all thePUs of a CU form a residual CU block for the transform component 304.

The transform component 304 performs block transforms on the residual CUto convert the residual pixel values to transform coefficients andprovides the transform coefficients to a quantize component 306. Thetransform component 304 receives the transform block sizes for theresidual CU and applies transforms of the specified sizes to the CU togenerate transform coefficients.

The quantize component 306 quantizes the transform coefficients based onquantization parameters (QPs) and quantization matrices provided by thecoding control component and the transform sizes. The quantize component306 may also determine the position of the last non-zero coefficient ina TU according to the scan pattern type for the TU and provide thecoordinates of this position to the entropy encoder 334 for inclusion inthe encoded bit stream. For example, the quantize component 306 may scanthe transform coefficients according to the scan pattern type to performthe quantization, and determine the position of the last non-zerocoefficient during the scanning/quantization.

The quantized transform coefficients are taken out of their scanordering by a scan component 308 and arranged sequentially for entropycoding. The scan component 308 scans the coefficients from the highestfrequency position to the lowest frequency position according to thescan pattern type for each TU. In essence, the scan component 308 scansbackward through the coefficients of the transform block to serializethe coefficients for entropy coding. As was previously mentioned, alarge region of a transform block in the higher frequencies is typicallyzero. The scan component 308 does not send such large regions of zerosin transform blocks for entropy coding. Rather, the scan component 308starts with the highest frequency position in the transform block andscans the coefficients backward in highest to lowest frequency orderuntil a coefficient with a non-zero value is located. Once the firstcoefficient with a non-zero value is located, that coefficient and allremaining coefficient values following the coefficient in the highest tolowest frequency scan order are serialized and passed to the entropyencoder 334. In some embodiments, the scan component 308 may beginscanning at the position of the last non-zero coefficient in the TU asdetermined by the quantize component 306, rather than at the highestfrequency position.

The ordered quantized transform coefficients for a CU provided via thescan component 308 along with header information for the CU are coded bythe entropy encoder 334, which provides a compressed bit stream to avideo buffer 336 for transmission or storage. The header information mayinclude the prediction mode used for the CU. The entropy encoder 334also encodes the CU and PU structure of each LCU.

The LCU processing includes an embedded decoder. As any compliantdecoder is expected to reconstruct an image from a compressed bitstream, the embedded decoder provides the same utility to the videoencoder. Knowledge of the reconstructed input allows the video encoderto transmit the appropriate residual energy to compose subsequentpictures. To determine the reconstructed input, i.e., reference data,the ordered quantized transform coefficients for a CU provided via thescan component 308 are returned to their original post-transformarrangement by an inverse scan component 310, the output of which isprovided to a dequantize component 312, which outputs a reconstructedversion of the transform result from the transform component 304.

The dequantized transform coefficients are provided to the inversetransform component 314, which outputs estimated residual informationwhich represents a reconstructed version of a residual CU. The inversetransform component 314 receives the transform block size used togenerate the transform coefficients and applies inverse transform(s) ofthe specified size to the transform coefficients to reconstruct theresidual values. The inverse transform component 314 may performtechniques for IDCT pruning as described herein.

The reconstructed residual CU is provided to the combiner 338. Thecombiner 338 adds the delayed selected CU to the reconstructed residualCU to generate an unfiltered reconstructed CU, which becomes part ofreconstructed picture information. The reconstructed picture informationis provided via a buffer 328 to the intra-prediction component 324 andto an in-loop filter component 316. The in-loop filter component 316applies various filters to the reconstructed picture information toimprove the reference picture used for encoding/decoding of subsequentpictures. The in-loop filter component 316 may, for example, adaptivelyapply low-pass filters to block boundaries according to the boundarystrength to alleviate blocking artifacts causes by the block-based videocoding. The filtered reference data is provided to storage component318.

FIG. 4 shows a block diagram of an example video decoder. The videodecoder operates to reverse the encoding operations, i.e., entropycoding, quantization, transformation, and prediction, performed by thevideo encoder of FIG. 3 to regenerate the pictures of the original videosequence. In view of the above description of a video encoder, one ofordinary skill in the art will understand the functionality ofcomponents of the video decoder without detailed explanation.

The entropy decoding component 400 receives an entropy encoded(compressed) video bit stream and reverses the entropy coding to recoverthe encoded PUs and header information such as the prediction modes andthe encoded CU and PU structures of the LCUs. If the decoded predictionmode is an inter-prediction mode, the entropy decoder 400 thenreconstructs the motion vector(s) as needed and provides the motionvector(s) to the motion compensation component 410.

The inverse scan and inverse quantization component 402 receives entropydecoded quantized transform coefficients from the entropy decodingcomponent 400, inverse scans the coefficients to return the coefficientsto their original post-transform arrangement, i.e., performs the inverseof the scan performed by the scan component 308 of the encoder toreconstruct quantized transform blocks, and de-quantizes the quantizedtransform coefficients. The forward scanning in the encoder is aconversion of the two dimensional (2D) quantized transform block to aone dimensional (1D) sequence; the inverse scanning performed here is aconversion of the 1D sequence to the two dimensional quantized transformblock using the same scanning pattern as that used in the encoder.

The inverse transform component 404 transforms the frequency domain datafrom the inverse scan and inverse quantization component 402 back to theresidual CU. That is, the inverse transform component 404 applies aninverse unit transform, i.e., the inverse of the unit transform used forencoding, to the de-quantized residual coefficients to produce theresidual CUs. The inverse transform component 404 may perform techniquesfor IDCT pruning as described herein.

A residual CU supplies one input of the addition component 406. Theother input of the addition component 406 comes from the mode switch408. When an inter-prediction mode is signaled in the encoded videostream, the mode switch 408 selects predicted PUs from the motioncompensation component 410 and when an intra-prediction mode issignaled, the mode switch selects predicted PUs from theintra-prediction component 414.

The motion compensation component 410 receives reference data fromstorage 412 and applies the motion compensation computed by the encoderand transmitted in the encoded video bit stream to the reference data togenerate a predicted PU. That is, the motion compensation component 410uses the motion vector(s) from the entropy decoder 400 and the referencedata to generate a predicted PU.

The intra-prediction component 414 receives reference data frompreviously decoded PUs of a current picture from the picture storage 412and applies the intra-prediction computed by the encoder as signaled bythe intra-prediction mode transmitted in the encoded video bit stream tothe reference data to generate a predicted PU.

The addition component 406 generates a decoded CU by adding thepredicted PUs selected by the mode switch 408 and the residual CU. Theoutput of the addition component 406 supplies the input of the in-loopfilter component 416. The in-loop filter component 416 performs the samefiltering as the encoder. The output of the in-loop filter component 416is the decoded pictures of the video bit stream. Further, the output ofthe in-loop filter component 416 is stored in storage 412 to be used asreference data.

FIG. 5 is a flow diagram of a method for scan adaptive selection of theinverse transform order, i.e., row-column or column-row, in a decoder.The decoder may be a standalone decoder such as that of FIG. 4 or partof an encoder such as that of FIG. 3 . More specifically, the methodselects the inverse transform order based on the scan pattern type usedduring encoding to scan the transform block. While the method isdescribed with five scan pattern types, more or fewer scan pattern typesmay be considered in other embodiments. The method assumes an N×N squaretransform block for simplicity of description. The method is describedwith reference to the examples of FIGS. 6-11 . In these examples, thenon-shaded regions of each block indicate the region of the respectiveblock containing only zero coefficients. Further, in these examples, theinput transform block is a 16×16 block and the position of the lastnon-zero coefficient in the transform block is indicated by an X.

Referring now to FIG. 5 , the position of the last non-zero coefficientin the transform block being decoded is determined 500. This positionmay be determined in any suitable way. For example, the position may beexplicitly signaled in the encoded bit stream. In HEVC, the x and ycoordinates of this position are available in the bit stream. A detaileddescription of signaling these x and y coordinates may be found, forexample, in HEVC Draft 6. Note that these coordinates are assumed to berelative to the upper left corner (0,0) of the transform block. If theposition signaled in the bit stream is relative to a larger block, thesignaled coordinates can be adjusted to be relative to the transformblock as the size of the block is known by the decoder. The variable LXis set to the X position and the variable LY is set to the Y position.The variable L is set to the minimum of N−1 and the sum of the Xposition and the Y position.

The scan pattern type is also determined 502. The scan pattern type maybe determined in any suitable way. For example, the scan pattern typemay be explicitly signaled in the bit stream and/or may be implicit inthe prediction mode used for the coding unit that includes the transformblock. In HEVC, the scan pattern type is implicitly signaled. Forexample, in HEVC, certain of the intra-prediction modes specify the scanpattern type. Thus, if the intra-prediction mode is known, the decodercan infer the scan pattern type. For Such intra-prediction modes aredescribed, for example, HEVC Draft 6. For the inter-predicted modes, thescan pattern is defined to be the fixed up right scan pattern.

Different inverse transform orders are then used to inverse transformthe block based on the scan pattern type. If the scan pattern type 502is horizontal, the row-column transform order is used. In a horizontalscan pattern, each row of the transform block is scanned sequentiallyfrom left to right. The arrows in the transform block 600 of FIG. 6illustrate this scan pattern. The computations of the 1D IDCT areperformed 504 on the first LY+1 rows of the transform block to generatethe first LY+1 rows of the interim results block, and the computationsof the 1D IDCT are then performed 506 on all N columns of the interimresults block to generate the final residual block.

FIG. 6 shows an example of the IDCT pruning when a horizontal scanpattern type is used and the transform order is row-column. In thisexample, the position of the last non-zero coefficient is (2,2).Accordingly, LY=2 and LY+1=3. That is, the Y coordinate+1 determines thenumber of rows for which the computations of the initial IDCT should beperformed. Thus, only the first 3 rows of the input transform block 600will be transformed to generate the interim results block 602. Note thatall rows of the interim results block 602 after the third row willcontain zeroes. Then, all columns of the interim results block 602 aretransformed with the 1D IDCT to generate the final residual block 604.

Referring again to FIG. 5 , if the scan pattern type 502 is vertical,the column-row transform order is used. In a vertical scan pattern, eachcolumn of the transform block is scanned sequentially from top tobottom. The arrows in the transform block 800 of FIG. 8 illustrate thisscan pattern. That is, the computations of the 1D IDCT are performed 508on the first LX+1 columns of the transform block to generate the firstLX+1 columns of the interim results block, and the computations of the1D IDCT are then performed 510 on all N rows of the interim resultsblock to generate the final residual block.

FIG. 7 shows an example illustrating why using row-column transformorder as in the prior art for a vertical scan pattern type isinefficient. If the transform order is fixed to be row-column, then allrows of the transform block 700 would be transformed by the 1D IDCT togenerate the interim results block 702. Then, all columns of the interimresults block 702 would be transformed by the 1D IDCT to generate thefinal residual block 704. Contrast this to the example of FIG. 8 .

FIG. 8 shows an example of the IDCT pruning when a vertical scan patterntype is used and the transform order is column-row. In this example, theposition of the last non-zero coefficient is (2,2). Accordingly, LX=2and LX+1=3. That is, the X coordinate+1 determines the number of columnsfor which the computations of the initial IDCT should be performed.Thus, only the first 3 columns of the input transform 800 will betransformed to generate the interim results block 802. Note that allcolumns of the interim results block 802 after the third column willcontain zeroes. Then, all columns of the interim results block 802 aretransformed with the 1D IDCT to generate the final residual block 804.

Referring again to FIG. 5 , if the scan pattern type 502 is zigzag and Lis not odd, then the row-column transform order is used. If X=0 or L=N−1522, then the computations of the 1D IDCT are performed 524 on the firstL+1 rows of the transform block to generate the first L+1 rows of theinterim results block, and the computations of the 1D IDCT are thenperformed 528 on all N columns of the interim results block to generatethe final residual block. If X≠0 and L≠N−1 522, then the computations ofthe 1D IDCT are performed 526 on the first L rows of the transform blockto generate the first L rows of the interim results block, and thecomputations of the 1D IDCT are then performed 528 on all N columns ofthe interim results block to generate the final residual block.

If the scan pattern type 502 is zigzag and L is odd, then the column-rowtransform order is used. If Y=0 or L=N−1 514, then the computations ofthe 1D IDCT are performed 516 on the first L+1 columns of the transformblock to generate the first L+1 columns of the interim results block,and the computations of the 1D IDCT are then performed 520 on all N rowsof the interim results block to generate the final residual block. IfY≠0 and L≠N−1 514, then the computations of the 1D IDCT are performed518 on the first L columns of the transform block to generate the firstL columns of the interim results block, and the computations of the 1DIDCT are then performed 520 on all N rows of the interim results blockto generate the final residual block.

FIG. 1 shows an example of the IDCT pruning when a zigzag scan patterntype is used and the transform order is row-column. In this example, theposition of the last non-zero coefficient is (4,4). Accordingly, L=8 andL+1=9. Since L is even, the row-column transform order is selected.Further, since LX≠0 and L≠N−1, only the first 9 rows of the inputtransform 100 will be transformed to generate the interim results block102. Note that all rows of the interim results block 102 after the ninthrow will contain zeroes. Then, all columns of the interim results block102 are transformed with the 1D IDCT to generate the final residualblock 104.

Referring again to FIG. 5 , if the scan pattern type 502 is down-left,then the row-column transform order is used. In a down-left scanpattern, each left diagonal of the transform block is scannedsequentially from top to bottom. The arrows in the transform block 900of FIG. 9 illustrate this scan pattern. If X=0 or L=N−1 522, then thecomputations of the 1D IDCT are performed 524 on the first L+1 rows ofthe transform block to generate the first L+1 rows of the interimresults block, and the computations of the 1D IDCT are then performed528 on all N columns of the interim results block to generate the finalresidual block. If X≠0 and L≠N−1 522, then the computations of the 1DIDCT are performed 526 on the first L rows of the transform block togenerate the first L rows of the interim results block, and thecomputations of the 1D IDCT are then performed 528 on all N columns ofthe interim results block to generate the final residual block.

FIG. 9 shows an example of the IDCT pruning when a down-left scanpattern type is used and the transform order is row-column. In thisexample, the position of the last non-zero coefficient is (4,5).Accordingly, L=9 and L+1=1. Since LX≠0 and L≠N−1, only the first 9 rowsof the input transform 900 will be transformed to generate the interimresults block 902. Note that all rows of the interim results block 902after the ninth row will contain zeroes. Then, all columns of theinterim results block 902 are transformed with the 1D IDCT to generatethe final residual block 904.

Referring again to FIG. 5 , if the scan pattern type 502 is up-right,then the column-row transform order is used. In an up-right scanpattern, each right diagonal of the transform block is scannedsequentially from bottom to top. The arrows in the transform block 1000of FIG. 10 illustrate this scan pattern. If Y=0 or L=N−1, then thecomputations of the 1D IDCT are performed 516 on the first L+1 columnsof the transform block to generate the first L+1 columns of the interimresults block, and the computations of the 1D IDCT are then performed520 on all N rows of the interim results block to generate the finalresidual block. If X≠0 and L≠N−1, then the computations of the 1D IDCTare performed 518 on the first L columns of the transform block togenerate the first L columns of the interim results block, and thecomputations of the 1D IDCT are then performed 520 on all N rows of theinterim results block to generate the final residual block.

FIG. 10 shows an example of the IDCT pruning when an up-right scanpattern type is used and the transform order is column-row. In thisexample, the position of the last non-zero coefficient is (4,5).Accordingly, L=9 and L+1=10. Since LY≠0 and L≠N−1, only the first 9columns of the input transform 1000 will be transformed to generate theinterim results block 1002. Note that all columns of the interim resultsblock 1002 after the ninth column will contain zeroes. Then, all rows ofthe interim results block 1002 are transformed with the 1D IDCT togenerate the final residual block 1004.

In the above method, when the diagonal scan is in the down-leftdirection as shown in FIG. 9 , the row-column transform order is used.When the diagonal scan is in the up-right direction as shown in FIG. 10, the column-row transform order is used. Note that if a row IDCT isapplied first for the up-right scan pattern of FIG. 10 , then 10 row 1DIDCTs would be performed rather than 9 column 1D IDCTs if the column-rowtransform order is used. In other words, if the direction of the lastdiagonal scan is known, the number of 1D IDCTs can be reduced by one.This can be extended to the zigzag scan pattern. The direction of thelast diagonal scan in a zigzag scan pattern can be determined from theposition of the last non-zero coefficient. If the sum of the coordinatesof the last position is even, e.g., [1,1] or [0,2], or [2,0], then thelast scan was in the up-right direction; otherwise, the last scan was inthe down-left direction. Thus, for the zigzag scan pattern, for optimalpruning, the transform order can be selected based on the position ofthe last non-zero coefficient as illustrated in the method.

The position of the last non-zero coefficient in a transform blockprovides a conservative estimate of the number of rows (columns) totransform when a zigzag, fixed up right, or fixed down left scan patternis used. Consider the example of FIG. 11 . Assuming a zigzag scanpattern, if the position of the last non-zero coefficient (denoted by X)is used to determine the number of rows (columns) to transform in thefirst 1D IDCT, six rows (7 columns) would be transformed. However, ifthe residual coefficients are distributed in a smaller region of thetransform block 1100 such as the 4×4 region 1101, then only four rows(columns) actually need to be transformed. The encoder can detect whenthe significant coefficients of a transform block are distributed in anM×N subset region of the transform block and signal such an occurrenceto the decoder. The decoder can then use this information to performmore optimal IDCT pruning.

The size(s) of the subset region(s), i.e., the values of M and N, may beset by the video coding standard in use and/or may be signaled at thetransform unit level, the coding unit level, the LCU level, the slicelevel, in a picture parameter set, and/or in the sequence parameter set.For example, the video coding standard may specify only one acceptablevalue for M and for N, in which case there would be no need to signalthe size to the decoder. In another example, the video coding standardmay specify multiple acceptable sizes for a subset region, in which casethe encoder would signal the specific size used at some level. In someembodiments, a specified size for subset region is M=N=4 as shown in theexample of FIG. 11 .

FIG. 12A is a flow diagram of a method for setting a pruned transformflag for a transform block in an encoder. Initially, the quantizedtransform block is scanned 1200 to determine if all non-zerocoefficients of the quantized transform block are in an M×N subsetregion of the transform block. Any suitable scanning technique may beused to make this determination. In some embodiments, the quantizedtransform block is scanned according to the scan pattern type for theblock to make this determination.

If all of the non-zero coefficients are in the M×N subset region 1202,the pruned transform flag is set 1208 to indicate this condition to thedecoder. The M×N subset region is then scanned to serialize 1210 thecoefficients to be entropy coded. The pruned transform flag and theserialized transform coefficients of the M×N subset region are thenentropy coded 1214 and added to the encoded bit stream.

As was previously described in reference to the scan component 308 ofFIG. 3 , quantized transform coefficients in a quantized transform blockare scanned to arrange them in serial order for entropy coding and toavoid entropy coding of large regions of zero value coefficients.Because it is known that all of the non-zero coefficients of thetransform block are in the M×N subset region, only this region needs tobe scanned to serialize the coefficients in the region and there is noneed to search for the first non-zero coefficient. The subset region isscanned according to the scan pattern type for the transform block.

If all of the non-zero coefficients are not in the M×N subset region1202, the pruned transform flag is set 1204 to indicate to the decoderthat this condition is not true. The quantized transform block is thenscanned 1206 according to the scan pattern for the block to serializethe coefficients to be entropy coded. The pruned transform flag and theserialized transform coefficients of quantized transform block are thenentropy coded 1214 and added to the encoded bit stream.

In some embodiments of the above method, the dimensions M and N of thesubset region are also encoded at the appropriate level in the encodedbit stream.

FIG. 12B is a flow diagram of a method for IDCT pruning in a decoder inwhich IDCT pruning may be performed responsive to a pruned transformflag. Initially, the pruned transform flag for a transform block isdecoded 1220. In embodiments where the dimensions of the M×N subset aresignaled, the dimensions are also decoded at whatever level they aresignaled. If pruned transform flag is true, then all the non-zerocoefficients of the transform block to be decoded are known to be in anM×N subset region of the block. The M×N quantized transform coefficientsare decoded from the encoded bit stream and inverse scanned 1224 torecreate the quantized transform block. The scan pattern type for thetransform block may be determined as previously described. Although notspecifically shown, the quantized transform block is de-quantized torecreate the transform block.

The transform block is then inverse transformed with pruning 1226 basedon one of the subset region dimensions to generate a residual block. Forexample, if M=N, then the number of rows (columns) to transform in thefirst 1D IDCT is M. In another example, the inverse transform order(row-column or column-row) may be selected based on the values of M andN. If M>N, then row-column order is selected and the first M rows of thetransform block are transformed to generate the interim results block.If M<N, then column-row order is selected and the first N columns of thetransform block are transformed to generate the interim results block.If M=N, a default inverse transform order is used, where the default iseither row-column order or column-row order.

If pruned transform flag is false, then all the non-zero coefficients ofthe transform block to be decoded are known not to be in an M×N subsetregion of the block. The quantized transform coefficients are decodedfrom the encoded bit stream and inverse scanned 1228 to recreate thequantized transform block. Although not specifically shown, thequantized transform block is de-quantized to recreate the transformblock. The transform block is then inverse transformed with regularpruning 1230 to generate a residual block. The regular pruning may beany suitable IDCT pruning technique, such as those previously describedherein. For example, the regular pruning may be performed according tothe scan pattern type used during encoding to scan the transform block.In another example, an embodiment of the method of FIG. 5 may be usedfor the regular pruning.

Each of the above IDCT pruning techniques can be successfully used toreduce computational complexity for transform blocks in which themajority of the block region is zero. In some embodiments, a furtherreduction in complexity may be achieved by pruning within the 1D IDCTsin each of these techniques to eliminate multiplication by 0 for thosecoefficients in the transform block or the interim results block knownto be zero. Consider the example of FIG. 1 . As was previouslyexplained, in this example, 1D IDCT computations would be performed ononly the first 9 rows of the transform block 100 to generate the interimresults block 102. Note that in for the first 9 rows of the transformblock 100, only a subset of the coefficients in each row may benon-zero; the remaining coefficients are known to be 0. For example, inthe first row, the last 6 coefficients are known to be zero and in the9^(th) row, all the coefficients but the first one are known to be 0.For each of the 9 1D IDCTs, the multiplications for the coefficientsknown to be 0 may be eliminated as the result of each is known to be 0.Similarly, the 1D IDCTS performed on the columns of the interim resultsblock 102 may be pruned within to eliminate the multiplications for thelast 7 coefficients of each column as these coefficients are known to bezero.

In the example of FIG. 6 , the 1D IDCT on the third row of the transformblock 600 may be pruned to eliminate the multiplications for the last 7coefficients of the row, and the 1D IDCTs on each of the columns of theinterim results block 602 may each be pruned to eliminate themultiplications for the last 13 coefficients of each column. In theexample of FIG. 8 , the 1D IDCT on the third column of the transformblock 800 may be pruned to eliminate the multiplications for the last 8coefficients of the column, and the 1D IDCTs on each of the rows of theinterim results block 802 may each be pruned to eliminate themultiplications for the last 13 coefficients of each row. In the exampleof FIG. 9 , the 1D IDCTs on the first 9 rows of the transform block 900may be pruned to eliminate the multiplications for coefficients known tobe zero in each row, and the 1D IDCTs on each of the columns of theinterim results block 902 may each be pruned to eliminate themultiplications for the last 7 coefficients of each column.

In the example of FIG. 10 , the 1D IDCTs on the first 9 columns of thetransform block 1000 may be pruned to eliminate the multiplications forcoefficients known to be zero in each column, and the 1D IDCTs on eachof the rows of the interim results block 1002 may each be pruned toeliminate the multiplications for the last 7 coefficients of each row.In the example of FIG. 11 , the 1D IDCTs on the first 4 rows of thetransform block 1100 may be pruned to eliminate the multiplications forthe last 12 coefficients in each row, and the 1D IDCTs on each of thecolumns of the interim results block 1102 may each be pruned toeliminate the multiplications for the last 12 coefficients of eachcolumn.

IDCT pruning in the context of a zigzag scanning pattern is a well-knowntechnique for reducing computational complexity in softwareimplementations of video coding standards. However, IDCT pruning is notknown to have been studied in the context of hardware implementation.Hardware implementations of video codecs are increasingly becomingessential to support high definition resolutions at high frame rates.Reduction of power consumption is very important because of surgingpopularity of mobile battery operated devices such as the iPad and othertablet computers and the iPhone and other smart phones.

Power consumption is closely related to switching activity, whichdetermines how often the capacitance is charged (when a signal goes from0→1). To maximize the impact of IDCT pruning in hardware, it isimportant to reduce any unnecessary switching that may be caused bysetting the inputs to zero. For instance, if the inputs (in binary) are1110→1010→1111, only the 2^(nd) and 4^(th) bit need to switched, i.e.,changed, from 0 to 1 when going from the 2^(nd) input to the 3^(rd)input. However, if with IDCT pruning, the 2^(nd) input is set to zero,then all 4 bits must be switched to go from the 2^(nd) input to the3^(rd) input (0000→1111).

In some embodiments, a hardware matrix multiplication architecture forIDCT pruning is provided that may reduce power consumption during videodecoding. In this architecture, pruning information, i.e., thecoefficients which are zero in a row or column of a transform block, isused to clock gates and disable computations that have zero input andzero output. FIG. 13 shows a simple example with two multipliers 1300,1302 and one adder 1304 which may be a part of a larger inversetransform circuit. The number of multipliers (and adders) in embodimentsof the architecture may vary and is a design decision.

In FIG. 13 , x0 and x1 are input coefficient values from a row (column)of the transform block and c0 and c1 are the IDCT coefficientscorresponding to the input values. The IDCT coefficients c0 and c1 areinputs to respective multipliers 1300, 1302. Individual prune flags,prune_flag0 and prune_flag1, are used to clock gate inputs to respectivemultipliers 1300, 1302. When x0 is zero, prune_flag0 is set to 1;otherwise, it is set to 0. When x1 is zero, prune_flag1 is set to 1;otherwise, it is set to 0.

When a prune_flag for x0 and/or x1 is set to 1, the clock is disabled atthe corresponding latch 1310, 1312. In this case, the previous values ofx0 and/or x1 are retained and there is no switching of circuits in therespective multipliers 1300, 1302. Because there is no switching, thereis no power consumption at one or both of the multipliers 1300, 1302(i.e., the multiplier is bypassed when the corresponding prune_flag is1). The outputs of the multipliers 1300, 1302 are muxed 1306, 1308 with0 with the selection made according to the respective prune_flag. If aprune_flag is 1, the output of the respective multiplexor 1306, 1308 is0, which is passed to the adder 1304.

When a prune_flag for x0 and/or x1 is set to 0, the clock is enabled atthe corresponding latch 1310, 1312, thus passing the current value of x0and/or x1 to the respective multiplier 1306, 1308. The multiplication isperformed and passed to the respective multiplexor 1306, 1308. If aprune_flag is 0, the output of the respective multiplexor 1306, 1308 isthe output of the respective multiplier 1300, 1302 which is passed tothe adder 1304. The adder 1304 adds the outputs of the multiplexors1306, 1308. Note that in architectures with more than pruningmultiplication circuits such as the ones of FIG. 13 , an adder tree maybe used combine the outputs of the pruning multiplication circuits. Theprune flags may be used in a fashion to similar that used to bypass themultiplication in order to bypass an adder in the tree if the output ofthe adder will be 0, i.e., if the prune flags of both inputs to theadder are zero.

FIG. 14 is a flow diagram of a method for pruning of zero valuecomputations in a matrix multiplication hardware architecture forperforming IDCT of transform blocks. This method may be performed foreach transform coefficient of the transform block. Further, the methodmay be performed in parallel for two or more transform coefficients inthe same row or column of the transform block. Initially, a transformcoefficient of the transform block and the corresponding IDCTcoefficient of an IDCT matrix are received 1400. If the transformcoefficient is 0 1402, a prune flag is set 1404 to indicate pruning ofthe multiplication operation. Otherwise, the prune flag is set 1406 toindicate no pruning of the multiplication operation.

If the prune flag indicates pruning 1408, latching of the transformcoefficient for multiplication with the IDCT coefficient is disabled1410 during a clock cycle, and a zero is output 1412. If the prune flagdoes not indicate pruning 1408, latching of the transform coefficientfor multiplication with the IDCT coefficient is enabled 1414 during aclock cycle, and the product of the transform coefficient and the IDCTcoefficient is output 1416.

Embodiments of the methods, encoders, and decoders described herein maybe implemented for virtually any type of digital system (e.g., a desktop computer, a laptop computer, a tablet computing device, a netbookcomputer, a handheld device such as a mobile (i.e., cellular) phone, apersonal digital assistant, a digital camera, a set top box, a digitalvideo recorder, etc.). FIG. 15 is a block diagram of a digital system1500 (e.g., a mobile cellular telephone) that may be configured to usetechniques described herein.

As shown in FIG. 15 , the signal processing unit (SPU) 1502 includes adigital signal processing system (DSP) that includes embedded memory andsecurity features. The analog baseband unit 1504 receives a voice datastream from the handset microphone 1513 a and sends a voice data streamto the handset mono speaker 1513 b. The analog baseband unit 1504 alsoreceives a voice data stream from the microphone 1514 a or 1532 a andsends a voice data stream to the mono headset 1514 b or wireless headset1532 b. The analog baseband unit 1504 and the SPU 1502 may be separateICs. In many embodiments, the analog baseband unit 1504 does not embed aprogrammable processor core, but performs processing based onconfiguration of audio paths, filters, gains, etc being setup bysoftware running on the SPU 1502.

The display 1520 may display pictures and video sequences received froma local camera 1528, or from other sources such as the USB 1526 or thememory 1512. The SPU 1502 may also send a video sequence to the display1520 that is received from various sources such as the cellular networkvia the RF transceiver 1506 or the Bluetooth interface 1530. The SPU1502 may also send a video sequence to an external video display unitvia the encoder unit 1522 over a composite output terminal 1524. Theencoder unit 1522 may provide encoding according to PAL/SECAM/NTSC videostandards.

The SPU 1502 includes functionality to perform the computationaloperations required for video encoding and decoding. In one or moreembodiments, the SPU 1502 is configured to perform computationaloperations for applying one or more techniques for IDCT pruning duringthe encoding process as described herein. Software instructionsimplementing all or part of the techniques may be stored in the memory1512 and executed by the SPU 1502, for example, as part of encodingvideo sequences captured by the local camera 1528. The SPU 1502 is alsoconfigured to perform computational operations for applying one or moretechniques for IDCT pruning as described herein as part of decoding areceived coded video sequence or decoding a coded video sequence storedin the memory 1512. Software instructions implementing all or part ofthe techniques may be stored in the memory 1512 and executed by the SPU1502.

OTHER EMBODIMENTS

While the invention has been described with respect to a limited numberof embodiments, those skilled in the art, having benefit of thisdisclosure, will appreciate that other embodiments can be devised whichdo not depart from the scope of the invention as disclosed herein.

For example, embodiments and examples have been described hereinassuming a square transform block. Transform blocks may also berectangular, e.g., 8×16, 16×8, 16×32, 32×16, etc. One of ordinary skillin the art will understand embodiments that include rectangulartransform blocks without need of further description.

Embodiments of the methods, encoders, and decoders described herein maybe implemented in hardware, software, firmware, or any combinationthereof. If completely or partially implemented in software, thesoftware may be executed in one or more processors, such as amicroprocessor, application specific integrated circuit (ASIC), fieldprogrammable gate array (FPGA), or digital signal processor (DSP). Thesoftware instructions may be initially stored in a computer-readablemedium and loaded and executed in the processor. In some cases, thesoftware instructions may also be sold in a computer program product,which includes the computer-readable medium and packaging materials forthe computer-readable medium. In some cases, the software instructionsmay be distributed via removable computer readable media, via atransmission path from computer readable media on another digitalsystem, etc. Examples of computer-readable media include non-writablestorage media such as read-only memory devices, writable storage mediasuch as disks, flash memory, memory, or a combination thereof.

It is therefore contemplated that the appended claims will cover anysuch modifications of the embodiments as fall within the true scope ofthe invention.

1. A method for decoding an encoded video bit stream in a video decoder,the method comprising: setting a prune flag to indicate pruning when avalue of a transform coefficient is zero; setting the prune flag toindicate no pruning when the value of the transform coefficient isnon-zero; disabling latching of the transform coefficient formultiplication with an inverse discrete cosine transformation (IDCT)coefficient during a clock cycle and outputting a zero when the pruneflag is set to indicate pruning; and enabling latching of the transformcoefficient for multiplication with the IDCT coefficient during a clockcycle and outputting a product of the transform coefficient and the IDCTcoefficient when the pruning flag is set to indicate no pruning.
 2. Themethod of claim 1, further comprising: determining a scan pattern typefor a transform block; decoding a position of a last non-zerocoefficient in the transform block; selecting a transform order based onthe scan pattern type; generating an interim results block based onperforming one dimensional (1D) IDCT computations on a subset of thetransform block according to the selected transform order; andgenerating a residual block using the interim results block.
 3. Themethod of claim 2, wherein selecting the transform order comprises:selecting a column-row inverse transform order when the scan patterntype is a first type; and selecting a row-column inverse transform orderwhen the scan pattern type is a second type.
 4. The method of claim 3,wherein the first type is a vertical scan pattern, and whereinperforming the 1D IDCT computations further includes: performing 1D IDCTcomputations on only a subset of columns of the transform block togenerate the interim results block, wherein a number of columns in thesubset is derived from a column position of the last non-zerocoefficient; and performing 1D IDCT computations on all rows of theinterim results block to generate the residual block.
 5. The method ofclaim 3, wherein the second type is a horizontal scan pattern; andwherein performing the 1D IDCT computations further includes: performing1D IDCT computations on only a subset of rows of the transform block togenerate the interim results block, wherein a number of rows in thesubset is derived from a row position of the last non-zero coefficient;and performing 1D IDCT computations on all columns of the interimresults block to generate the residual block.
 6. The method of claim 3,wherein the first type is an up-right scan pattern, and whereinperforming the 1D IDCT computations further includes: performing 1D IDCTcomputations on only a subset of columns of the transform block togenerate the interim results block, wherein a number of columns in thesubset is derived from a column position and a row position of the lastnon-zero coefficient; and performing 1D IDCT computations on all rows ofthe interim results block to generate the residual block.
 7. The methodof claim 3, wherein the second type is a down-left scan pattern, andwherein performing the 1D IDCT computations further includes: performing1D IDCT computations on only a subset of rows of the transform block togenerate the interim results block, wherein a number of rows in thesubset is derived from a row position and a column position of the lastnon-zero coefficient; and performing 1D IDCT computations on all columnsof the interim results block to generate the residual block.
 8. Themethod of claim 3, wherein the first type is a zigzag scan pattern, andwherein performing the 1D IDCT computations further includes: performing1D IDCT computations on only a subset of columns of the transform blockto generate the interim results block, wherein a number of columns inthe subset is derived from a column position and a row position of thelast non-zero coefficient; and performing 1D IDCT computations on allrows of the interim results block to generate the residual block.
 9. Themethod of claim 3, wherein the second type is a zigzag scan pattern, andwherein performing the 1D IDCT computations further includes: performing1D IDCT computations on only a subset of rows of the transform block togenerate the interim results block, wherein a number of rows in thesubset is derived from a row position and a column position of the lastnon-zero coefficient; and performing 1D IDCT computations on all columnsof the interim results block to generate the residual block.
 10. Themethod of claim 3, wherein selecting the column-row inverse transformorder comprises selecting the column-row inverse transform order when:the scan pattern type is a zigzag scan pattern; and a scan direction ofa diagonal of the transform block in which the last non-zero coefficientis located is up-right.
 11. The method of claim 3, wherein selecting therow-column inverse transform order comprises selecting the row-columninverse transform order when: the scan pattern type is a zigzag scanpattern; and a scan direction of a diagonal of the transform block inwhich the last non-zero coefficient is located is down-left.
 12. Anon-transitory computer-readable medium having executable instructionsstored thereon, configured to be executable by processing circuitry forcausing the processing circuitry to: set a prune flag to indicatepruning when a value of a transform coefficient is zero; set the pruneflag to indicate no pruning when the value of the transform coefficientis non-zero; disable latching of the transform coefficient formultiplication with an inverse discrete cosine transformation (IDCT)coefficient during a clock cycle and outputting a zero when the pruneflag is set to indicate pruning; and enable latching of the transformcoefficient for multiplication with the IDCT coefficient during a clockcycle and outputting a product of the transform coefficient and the IDCTcoefficient when the pruning flag is set to indicate no pruning.
 13. Thenon-transitory computer-readable medium of claim 12, wherein theinstructions are executable by the processing circuitry for furthercausing the processing circuitry to: determine a scan pattern type for atransform block; decode a position of a last non-zero coefficient in thetransform block; select a transform order based on the scan patterntype; generate an interim results block based on performing onedimensional (1D) IDCT computations on a subset of the transform blockaccording to the selected transform order; and generate a residual blockusing the interim results block.
 14. The non-transitorycomputer-readable medium of claim 13, wherein the instructions to selectthe transform order comprise instructions to: select a column-rowinverse transform order when the scan pattern type is a first type; andselect a row-column inverse transform order when the scan pattern typeis a second type.
 15. The non-transitory computer-readable medium ofclaim 14, wherein the first type is an up-right scan pattern, andwherein the instructions to perform the 1D IDCT computations compriseinstructions to: perform 1D IDCT computations on only a subset ofcolumns of the transform block to generate the interim results block,wherein a number of columns in the subset is derived from a columnposition and a row position of the last non-zero coefficient; andperform 1D IDCT computations on all rows of the interim results block togenerate the residual block.
 16. The non-transitory computer-readablemedium of claim 14, wherein the second type is a down-left scan pattern,and wherein the instructions to perform the 1D IDCT computationscomprise instructions to: perform 1D IDCT computations on only a subsetof rows of the transform block to generate the interim results block,wherein a number of rows in the subset is derived from a row positionand a column position of the last non-zero coefficient; and perform 1DIDCT computations on all columns of the interim results block togenerate the residual block.
 17. A method for decoding an encoded videobit stream in a video decoder, the method comprising: decoding a prunedtransform flag in the encoded video bit stream, wherein the prunedtransform flag indicates that a subset region of a transform blockcontains all non-zero coefficients of the transform block, wherein thesubset region consists of M rows and N columns of the transform block;and inverse scanning only quantized transform coefficients of the subsetregion decoded from the encoded video bit stream to generate thetransform block.
 18. The method of claim 17, wherein M=N=4.
 19. Themethod of claim 17, further comprising: performing one dimensional (1D)inverse discrete cosine transformation (IDCT) computations on thetransform block to generate a residual block, wherein one of a value ofM or N is used to determine pruning of the 1D IDCT computations.
 20. Themethod of claim 19, wherein performing 1D IDCT computations comprises:using a column-row inverse transform order when N>M, wherein 1D IDCTcomputations are performed on only N columns of the transform block togenerate an interim results block; and using a row-column inversetransform order when M>N, wherein 1D IDCT computations are performed ononly M rows of the transform block to generate the interim resultsblock.